Tracking systems and methods are known in the art. In general, a tracking system is used to calculate and determine the position and orientation of an object relative to a given coordinate system in a platform. The tracked object can be, for example, a medical probe relative to a human body, the head of a pilot relative to the cockpit of a fighter jet, a person relative to a room of a building, or a ship relative to a body of water. The platform can be, for example, the human body, the cockpit of a fighter jet, a room of a building, or an entire planet. In general, tracking systems can be classified into two main categories namely, inertial trackers and non-inertial trackers.
Inertial trackers were first developed in the early twentieth century as an aid in marine navigation and later in aerial navigation. Whereas inertial trackers have grown in complexity since they were first invented, and are now used for a variety of different applications (i.e., navigational and other), the general principles behind inertial tracking remain the same. Inertial trackers essentially include a device, referred to in the art as an inertial sensor, capable of continuously measuring the linear acceleration {right arrow over (a)}, and the angular velocity {right arrow over (ω)}, of an object relative to an inertial reference frame. The linear acceleration {right arrow over (a)} may be measured using an accelerometer, and the angular velocity {right arrow over (ω)} may be measured using a gyroscope. It is noted that through mathematical integration techniques known in the art, the position and orientation of the object can be determined from {right arrow over (a)} and {right arrow over (ω)} by integrating the equations of motion. Inertial tracking requires that the initial position and orientation of an object be recorded and stored. {right arrow over (a)} and {right arrow over (ω)} of the object are typically measured at rapid intervals. Since the initial position and orientation of the object is known, using the equations of motion, the dynamic position and orientation of an object can be continuously predicted, based on the mathematical integration of {right arrow over (a)} and {right arrow over (ω)} measurements. Therefore, given a previous estimate of the position and orientation of an object, the current position and orientation of the object can be determined by integrating the equations of motion based on newly measured values of {right arrow over (a)} and {right arrow over (ω)}.
Inertial trackers incorporate two additional requirements. The first requirement is an accurate model of the inertial sensor, so that raw data measured from the inertial sensor can be corrected for bias, scale factor and non-orthogonality. The second requirement is an estimation of the position and orientation of the object at sufficiently close time intervals. Accordingly, when the second requirement is met, one can assume that the current position and orientation of the object are determined by {right arrow over (a)} and {right arrow over (ω)} alone, and not in addition to any other higher-order derivates of the quantities of motion, such as jerk, {right arrow over ({dot over (a)} (the rate of change of acceleration) and rotational acceleration, {right arrow over (α)}, where {right arrow over (α)} is the derivative of {right arrow over (ω)}. If the initial position and orientation of the object are not known, then an inertial tracker cannot determine the dynamic position and orientation of the object. If either of the two aforementioned requirements is not met, then the position and orientation determined by an inertial tracker may be inaccurate.
Furthermore, inertial trackers suffer from drift. Due to the noise or jitter in the measurements of the inertial tracker, the integration of these measurements over a period of time results in a drift error, which gradually increases the inaccuracy of the final results. Drift is usually corrected by using very expensive gyroscopes or accelerometers to implement the inertial tracking measurements.
Electromagnetic (EM) tracking operates on a different principle than inertial tracking. In general, electromagnetic tracking involves determining the position and orientation of an electromagnetic receiver relative to an electromagnetic source. An electromagnetic source generates an electromagnetic field within a platform (e.g., the cockpit of an airplane). The electromagnetic source includes at least one electromagnetic coil that transmits an electromagnetic field. For example, the electromagnetic source may include three mutually perpendicular transmitting coils, each defining a respective axis. An electromagnetic receiver unit on the tracked object (e.g., the helmet of the pilot) measures the generated electromagnetic field. The electromagnetic receiver unit includes at least one sensor (e.g., three) that measures values associated with the electromagnetic field. Each sensor includes at least one electromagnetic coil that receives an electromagnetic field. For example, each sensor may include three mutually perpendicular receiving coils, each defining a respective axis. The electromagnetic source generates a unique electromagnetic field, in the sense that each point in the motion platform has a unique measurable quantity associated with the electromagnetic field (e.g., the electromagnetic field strength as a vector quantity). The position and orientation of the sensor is obtained by analyzing and comparing the electromagnetic field generated by the transmitting coils and the electromagnetic field measured by the receiving coils.
One approach for EM tracking is based on using a mathematical model of the generated electromagnetic field. The mathematical model is characterized by a set of model parameters, with one set for each transmitting coil. The model predicts the electromagnetic field value (i.e., the measurable quantity associated with the electromagnetic field) at each point within the platform. The EM tracker solves for the position and orientation at which the electromagnetic field values predicted by the mathematical model of the electromagnetic field best match the field values measured by the receiving coils.
A cost function defines the difference between the measured EM field and the model. By minimizing the cost function using a best-fit algorithm, the EM tracker finds the closest match between the model of the EM field and the measured EM field. Noise and other factors influence the final result.
The calculation of position and orientation in EM tracking in this manner necessitates two requirements. The first requirement is that the EM tracker possesses an accurate model of the EM field transmitted by the electromagnetic source, for every point within the platform. It is noted that there are a number of methods known in the art for obtaining all the necessary model parameters, such as analytical modeling (as described in F. Raab, E. Blood, T. Steiner and H. Jones, “Magnetic Position and Orientation Tracking System”, IEEE Transactions on Aerospace and Electronic Systems, 1979, Vol. AES-15, No. 5, pp. 709-717; and U.S. Pat. No. 3,868,565 to Kuipers, entitled “Object Tracking and Orientation Determination means, System and Process”), and adaptive modeling (as described in US Patent Application No. 2004/0034515 A1 to Bar Tal et al, entitled “Estimating Position and Orientation in Electromagnetic Systems”). The second requirement is that the EM tracker possesses an accurate model of the receiving coils (i.e., size, shape, and other physical parameters of the coils). This model is a rigid structure characterized by the vector position of each sensor in the electromagnetic receiver unit, and the calibration parameters of each sensor.
Several EM trackers suffer from latency. Typically, in order to reduce the noise in the EM field measurements, a large number of previous measurements are averaged. This procedure results in a substantial delay, or latency, between the position and orientation determined by the EM tracker, and the actual current position and orientation of the tracked object.
It is noted that EM tracking is a special case of the broader category of non-inertial tracking. Other forms of non-inertial tracking, such as optical, acoustic, or radio frequency (RF), involve generating and receiving some form of energy or signal (i.e., light waves, sound waves, or radio frequencies, respectively).
Non-inertial trackers may be influenced by external factors in the platform environment. For example, EM trackers may be influenced by electrical or magnetic objects in the environment, which can interfere with the electromagnetic field generated by the electromagnetic source, or with the electromagnetic field received by the electromagnetic receiver unit. Another example involves optical trackers, where objects near the tracked object may block a direct line-of-sight from a sensor to a transmitter, and prevent the sensor from receiving the transmitted optical signal. These types of interference may result in limitations for the environment in which non-inertial tracking is implemented.
U.S. Pat. No. 5,645,077 to Foxlin entitled “Inertial Orientation Tracker Apparatus Having Automatic Drift Compensation for Tracking Human Head and Other Similarly Sized Body”, is directed to an inertial angular orientation tracking apparatus which utilizes additional sensors for drift correction. A tracking device includes a set of drift sensitive sensors, a set of drift compensating sensors, an integration module, and a drift compensation module. The drift sensitive sensors are typically angular rate sensors, such as rate gyroscopes, and are subject to drift over time. The drift compensating sensors are angular position sensors, typically inclinometers and compasses, and are relatively impervious to drift over time. The angular rate sensors and drift compensating sensors are mounted upon a human body member, such as a head. The angular rate sensors and drift compensating sensors are impervious to interference from electromagnetic, acoustic, optical and mechanical sources.
The set of angular rate sensors generate signals corresponding to the rate of change of the angular orientation of the individual sensors. The integration module integrates the signals, to provide a set of orientation angles. The set of drift compensating sensors generate an angular position signal related to the angular orientation of the body member to which the sensors are attached. The angular position signal is provided to the drift compensation module. The drift compensation module uses the angular position signal to generate a signal representing the angular orientation corrected for errors in the angular orientation provided by the integration module. These errors may arise due to drift. Drift results from integrating over time a signal which is noisy or has a bias. Drift would make the virtual world appear to gradually rotate about the head of the user even when the user is not moving. The corrected angular orientation signal is provided as output to external equipment, such as a virtual reality system, a computer, or a teleoperator.
The drift compensating module may include a verifier that periodically measures the orientation of the body member, by a means different from the angular rate sensors. The verifier generates an orientation drift compensation signal based on the verification measurement, to reduce the effect of drift. The verifier may take into account characteristic features of human motion, such as stillness periods. The drift compensating module may be implemented using a Kalman filter, which may utilize statistical data about human head motion.
U.S. Pat. No. 6,409,687 to Foxlin entitled “Motion Tracking System”, is directed to an inertial motion tracking apparatus which utilizes acoustic measurements for drift correction. A tracking device includes an Inertial Measurement Unit (IMU), an array of three Ultrasonic Range Measurement (URM) Units, and a processor. The processor is coupled the IMU and the URMs. The tracking device may be fixed on the head of an operator, such as to a Head-Up Display (HUD).
The IMU includes three angular rate sensors and three linear acceleration sensors. The sensors are arranged along three orthogonal axes that are fixed in the reference frame of the tracking device. Each acceleration sensor provides a signal proportional to the acceleration along the corresponding axis. Each angular rate sensor provides a signal proportional to the rotation rate about the corresponding axis. The processor determines the position and orientation of the tracking device using the inertial measurements received from the IMU.
An ultrasonic beacon array includes a plurality of ultrasonic beacons at fixed locations on the environment, such as arranged on the ceiling of the room in a regular pattern. The URM receives acoustic signals sent by the ultrasonic beacons. The processor estimates a range measurement relating to the distance between the beacons and the tracking device, using the received signals along with the known three-dimensional location of the beacons. The processor updates the position and orientation of the tracking device using the range measurements.
Each ultrasound beacon sends an acoustic pulse to the URMs. Each URM broadcasts infrared (IR) signals to all the ultrasonic beacons. The IR signals include address information, so that only a selected number of ultrasound beacons respond. In response to an IR signal, the intended ultrasonic beacons immediately send acoustic pulses, which are received by one or more URMs. The processor determines the time of flight, by measuring the delay from issuing the IR command to detecting the ultrasonic pulse. The range measurement may be based on the time of flight.
The tracking device combines measurements from both measurement modes, and adjusts its estimates of position and orientation to reflect measurements from both modes as they are available. The processor may implement an extended Kalman Filter to combine the measurements and maintain ongoing estimates of the position and orientation of the tracking device, along with the degree of uncertainty of these estimates.
The tracking device continues using inertial measurements when acoustic measurements are unavailable, such as due to occlusion of the beams. Drift correction is then made once the acoustic measurements become available.
U.S. Pat. No. 6,474,159 to Foxlin et al. entitled “Motion-Tracking”, is directed to an inertial tracker for tracking motion relative to a moving platform. An inertial head-tracking system on board a moving platform includes a tracking Inertial Measurement Unit (IMU), a reference IMU, and a processor unit. The tracking IMU is mounted on the tracked object. The reference IMU is mounted on the moving platform, such as on an external X-bar fixed to the environment (e.g., the canopy of the simulator cab or the cockpit).
Each of the reference IMU and the tracking IMU may include three angular inertial sensors having angular rate sensors. The processor unit receives signals from the tracking IMU and the reference IMU, and determines the orientation of the object relative to the moving platform. In particular, the processor unit computes the orientation of the object with respect to a fixed inertial reference frame using the signals from the tracking IMU. The processor unit further computes the orientation of the moving platform with respect to the same fixed inertial reference frame using the signals from the reference IMU. Based on these two orientations, the processor unit then computes the orientation of the object relative to the moving platform. The orientation relative to a given reference frame is determined by integrating a relative angular rate signal measured by the IMU.
A non-inertial tracking subsystem may make independent measurements of the orientation of the object relative to the moving reference frame. The inertial head-tracking system may use these measurements to correct for any drift that may occur in the inertial orientation integration. The non-inertial tracking subsystem may use optical, acoustic, magnetic, radio frequency, or electromagnetic technologies.
Each of the reference IMU and the tracking IMU may further include three linear accelerometers. The processor unit may further determine the position of the object relative to the moving platform, using signals from the tracking IMU and the reference IMU. In particular, the processor unit computes the position of the object with respect to a fixed inertial reference frame using the signals from the tracking IMU. The processor unit further computes the position of the moving platform with respect to the same fixed inertial reference frame using the signals from the reference IMU. Based on these two positions, the processor unit then computes the position of the object relative to the moving platform. The position relative to a given reference frame is determined by double-integrating a relative linear acceleration signal measured by the IMU.
The calculation of the linear acceleration signal may include compensation for tangential, Coriolis, and centripetal acceleration effects caused by the angular velocity and angular acceleration of the moving platform. The non-inertial tracking subsystem may further make independent measurements of the position of the object relative to the moving reference frame. The inertial head-tracking system may then use these measurements to correct for any drift that may occur in the inertial position integration.
US Patent Application No. 2004/0073360 A1 to Foxlin entitled “Tracking, auto-calibration, and map-building system”, is directed to a method and system for navigating a vehicle in a certain environment. The navigation system builds a map of the environment, calibrates the sensors, and determines the location of the vehicle in the environment frame of reference. A navigation system includes sensors and a data processing unit. The sensors provide measurement data. The data processing unit processes the data provided by the sensors.
The system includes a sensor subsystem and an estimation subsystem coupled thereto. The sensor subsystem includes inertial sensors and sensing devices or targets. The sensing devices or targets, referred to as Pose Sensing Elements (PSEs), may be in the vehicle (referred to as “mobile PSE”), or fixed in the environment (referred to as “fixed PSE”). The inertial sensors provide inertial measurements to the navigation system. The PSEs provide sensor measurements, which are measurements between elements in the vehicle and elements external to the vehicle. Sensor measurements relate to measurements between a mobile PSE and a fixed PSE, and depend on aspects of the relative pose of the two PSEs.
The estimation subsystem accepts configuration data and measurement information from the sensor subsystem. The measurement information relates to the relative position and orientation of sensing elements. The configuration data enables selective performance of different functions for the estimation subsystem. The estimation subsystem repeatedly updates a state estimate, which characterizes an estimate of the location of the vehicle, according to the accepted measurement data and configuration data. The state estimate may be an estimate of the position or orientation of the vehicle relative to the environment, the position and orientation, relative to the vehicle, of a sensing device fixed to the vehicle, or the position and orientation, relative to the environment, of a sensing device fixed in the environment. Updating the state estimate may be performed using a Kalman filter.